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ABSTHACT 

A aodel for the evaluator-pcllcy aaker relationship 
is presented for use by those responsible, fer conducting progran 
evaluations which ultiaately affect policy decisions. In this 
collegial aodel, power is invested equally in relevant research and 
prograa^personnel. Characteristics of the acdel are described in 
terns of; (1) early contact with the prograa; (2) willingness to 
learn about the pfrograa;’ (3) a consultive approach to'evaluation 
design: (4) regular contact with the prograa; and (5) aforaal 
nechanisB for proqraa personnel feedback. An application cf this 
aodel is provided through a descriptict cf the Aaerican Ear 
Association BASICS Prograa. (Author/BLB) 
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relationship, between the researchers and the policy-maker. It Is my conten- 

t 


tion that, adose and on-goihg relationship between researcher and program 
personnel of all“types must occur ff research results are to be relevant and 
timely, and hence, ready for utilization. I propose a collegial model for the 
evaluator-policy maker relationship, in contrast to others who do not encour¬ 


age a close relationship (e.g., Michael Scriven, 1976). I would like to 

-• •*> 
Illustrate the model first by describing the evaluation of a nationwide cor¬ 


rectional reform program which I have conducted over the past four years. Then, 
I will highlight components of the model, using examples from this research 
project. y 




Psychologists, as well as other social scientists,, are becoming Increa^ 
singly Involved in policy'research. Following Campbell's {TS^).plea for an. 
experimental approach to reform, sonie social psychologists emerged from their* 
-methodological closets and began to apply their .research skills to the eval¬ 
uation of.social* reform programs. The profession of evaluation researches 

■ ■ . - •. i 

* v >. . . 

grown rapidly in the past few years, largely on the promise that psycholo- 

, ’ * ' . ’ * * * * 
gists and other evaluation researcher^ co^d. provide data on program effects 

tl£policy-makers. The assumption ^as that policy-makers, once armed with con¬ 
clusive data, would make more rjitiphal, better-informed decisions. Better 
knowledge would automatically/ \j[ seemed, result in better policy. 

This result, alas, has no^ necessarily occurred. The fauU has not been 
so much in the quality of the data evaluation researchers have collected 
(although new evaluation designs and new data-collection tools are still needed) 
but rather .In the decisions made about which aspects of the program to research 
or about how apd when to present the findings. These decisions arise from the 








In eacly 1974, the American Bar Association (ABA)‘ began a new program 

j ■ 1 , „ „ ■ 

.In criminal justice reform. The program was developed by the ABA's Commission 
on Correctional Facilities and Services and was named BASICS, an acronym which 
stands for Bar Association Support to Improve Correctional Services. As the 
name Implies, the BASICS Program was intended to improve correctional*services 
using a new method: bar association involvement'and work on local\*4m1na1 * 
justice problems. This novel method for effecting correctional reformjfvas 
developed by the Staff Director of the ABA's Commission on Correctional Facll* 
Itles and Services and by the Vice-President of the Edna McConnell Clark Foun¬ 
dation, which funded the new program. These men believed that lawyers and 

i 

the bar associations of which they were members were, untapped resources in the 

•• 

correctional reform area .L * 

The BASICS plan, then, was to activate local bar association members 

first to plan (the "planning j)h a se" ) and later to implement* (the "action phase") 

\ ' 

some type of correctional reform effort. The project solicited applications 
from bar associations across the country for small planning grants df.approx¬ 


imately $3000. From the approximately 1000 state, county and local bar assoc¬ 
iations across the country, BASICS received 106 applications and funded 80 , 
planning projects In 40 states (Huff,. Conner and Geis, 1975). After approx¬ 
imately three months of planning, 62 bar associations applied for larger 
grants up to $35,000 to implement their correctional reform efforts. Twenty 

"action" grants were awarded by the BASICS Program. 

. / 

The. BASICS Evaluation 

Because of their interest in effecting "measurable changes, 1 * the BASICS 
Management Board and staff began to discuss program evaluation at the outset 
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Of the Program. 1 They asked our'interdisciplinary team of evaluators (a social 
psychologist with an Interest-in evaluation research and a sociologist with’ 
an Interest In criminology) to formulate an evaluation plan. The evaluation 
design had to conform to the Program’s unusual operational procedure which 

Involved a planning phase, during which bar associations formulated correc- ' 

. • ' , • • • • ' x rs, 

tlonal reform plans, followed/by an action phase, during whlch’the associa¬ 
tions were to Implement thetr projects. The evaluation plan for tlie first * 
phase involved a survey of funded mnd non-funded bar^assocjations, Interviews 
with a sample of funded associations, as well as Interviews with the BASICS 
staff'arfcd' Clark Foundation;representatives. The product of this phase of the 
evaluation was a report to the Board of Directors containing data on the more 
and less successful aspects of the first phase and recojnmendations for the 
future (Huff, Conner and. i^eis, 1975). 

The second phase of the project required a different evaluation plan. 
Because 20 bar associations'were funded to implement 20 different kinds of 

evaluations projects, no single evaluation design would suit all projects. 

. i 

Instead, the BASICS Management Board, staff and research tea‘m jointly selected 

^ - 

four projects for intensive evaluation.* These four were selected primarily 
because the interventions had the potential to be modeled by other bar pro¬ 
jects. The Jfour projects were a prison legal services a<od-pa rale gal training 
project in Michigan (Conner, Emshoff and Davidson^ 1978), a citizen dispute 
settlement program in Florida (Conner and*Surette, 1977), a jail legal ser¬ 
vices project In.Maryland (Huff’, 1 978) a*nd an offender vocational training 

i > « < , 

■project, also in Maryland CHuff, 1978). Each member of the research team 
worked closely with two reform projects to develop and implement an appropriate 
evaluation plan. The designs selected ranged from a control-group study to an 


. At both the macro policy-making level, with the Board of Directors and 0 

# . 3 - • "m ‘ 

* ’ . * 

.theprogram manager, and at the micro policy-making level, with director^f 
the four Individual programs, we used the same'approach to the evaluator- 

t 

program relationship: a‘ collegial, cooperative model. These activities-char- 
, ’ * . • . / 
4 cterized our approach: (1) we began to work with the program personnel be¬ 
fore the program began; (2) we quickly learned as much as we could about the/"* . 
program and kept learning throughout the course of the researchr (3) we.planned 
the research with the program personnel's help; (4) we -thoroughly presented 

'* "N 

and discussed our final evaluation design, measures and time-table with the 
program personnel* (5) we maintained regular contact with the program, in ,two 
cases hiring evaluation research assistants who maintained almost daily*con¬ 
tact; and (6), as promised at the outset of the research, we presented the prog- 

t 

ram personnel with drafts of our reports for their comments, which were included 
with the final, published report. 

The collegial model of the evaluator-program relationship, then, is char¬ 
acterized by (1) early contact with the program, ideally prior to the start 
of the program; (2) a willingness to learn about the program and keep learning 
as the program is implemented; (3) a consultive approach to evaluation design t 
where the evaluator has full and final responsibility for-designing the study, 
choosing the measures and setting the timetable but does so with the advice 

if 

of the program personnel; (4) regular contact , ideally in person, with the 
jirogram; and (5) a formal mechanism for program personnel feedback on the eval¬ 
uation resuTts. 
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The personal styles of the twa evaluation researchers was -also Important. 
Bdth^f us had been Involved In social reform work (one of us was In the Peace 
Corps arid the other was ^ social worker), so we could understand the program s 


personnels^' concerns. I-n addition, we believe there was a mutual respect 


for the different skills each of us—evaluators and program personnel alike— 


i5i*8ught to the program and the different tasks each had to accomplish. Rather^ 


than highlight our differences from the^program personnel, we as researchers 


v. 


Instead tried to maximize the similarities and to take actions to mlnlmlz'e the 
differences where this was feasible (e.g., learning about the'program,*the 
community, the staff, the directors; spending time experiencing day-to-day 


activities of the program). 

It is worthwhile questioning whether this collegial model we followed was 


really the cause of the^cooperative relationship we experienced. Perhaps the 


' t 


approach was not the causal factor:- this just might have been one of those 
. * 

rare instances where—by chance—all worked for the best. Several things 


would lead us to doubt .this. First, in the five evaVuator-program personnel 


relationship, there were degrees of cooperativeness. In those cases where 
-we experienced our best relationship, the six activities notedabove were 
carried out without exception. Where we were not able to implement all of 
these activities, the evaluator-program relationship was less satisfactory. 
Second, the reaspns for the inability to implement the activities fully were 

attributable more to the evaluators than td the program. For example, one of 

/ ' 

us was unable to maintain close contact with a project due to an illness, and 
this affected the evaluation. ’ Third, these were different kinds of projects 
involving different- kinds of people. Yet, we generally were, able to imple¬ 
ment our evaluation plap in spite of these differences* ' • 
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*e experienced. Itiv fact,;irre: 
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*•* ® synergistic quality-improving ttexonie^^ 

; ■ .. v ' *j _ ■ ' .-* , ^ >£** :^j 

fS pfogra*as well as of the conduct of-the evaluation. The improvementof the - 
content of the programs occurred, first, via formative evaluation results ari&'. v 
discussions, then later, via the suramatlve evaluattoltyresul ts. At the national 't 

level, our results on the planning phase affected the c^plept and direction 

. V . ' _• . V.. ; ■ > 

of the action phase. At the local level, the conduct of the evaluation was 
Improved because the program directors aided In the implementation of the eval¬ 
uation, thereby helping raise client participation*. There was one additional ’ 

unintended effect which we were pleased to note: after we were* no longer 

\ . f ‘ .... ..... 

associated with the Individual projects, two of the four project directors 

. -t 

Implemented the same research plan for a second year (Including, In one case, 

— ' j ' ’ : * 

a control-group research design) because of the utility of the data they rec- 
. elved at the end of the first year. This kind r t^f unobtrusive measure of sati¬ 
sfaction with the research provides additional support for a collegial evaluator- 
* * 

program relatldhship, as well as a concrete indication that the bridge between 
the researchers and the program personnel had been established. 
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